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1 . 0 Introduction  and  Document  Scope 

The  Standard  for  the  Exchange  of  Product  Model  Data  (STEP)  is  an 
evolving  international  standard  developed  under  sponsorship  of  the 
International  Organization  for  Standardization  (ISO).  The 
objective  of  the  STEP  standard  is  to  provide  a complete, 
unambiguous,  computer  interpretable  definition  of  the  physical  and 
functional  characteristics  of  a product  throughout  its  life  cycle. 

PDES  (Product  Data  Exchange  Using  STEP)  is  the  name  given  to  the 
United  States  development  activity  in  support  of  this  international 
standard.  In  order  to  accelerate  the  development,  validation,  and 
implementation  of  STEP,  a joint  industry/government  consortium  was 
formed  in  1988  and  incorporated  in  the  state  of  Delaware  as  PDES, 
Inc.  South  Carolina  Research  Authority  (SCRA)  has  been  awarded  the 
Host  Contract  to  provide  technical  management  for  the  PDES,  Inc. 
program. 

The  methodology  used  by  PDES,  Inc.  to  validate  the  STEP  standard 
consists  of  testing  the  STEP  resource  models  (i.e.,  information 
data  models)  from  a very  specific  application  point  of  view.  This 
application  viewpoint  reguires  development  of  a Context-Driven 
Integrated  Model  (CDIM).  A CDIM  is  an  integrated  model  composed  of 
data  definitions  and  data  construct  relationships,  constraints, 
etc.  from  the  STEP  resource  models,  which  satisfy  data  regui.rements 
for  a specified  application  context.  The  National  Institute  of 
Standards  and  Technology  (NIST)  has  contributed  to  the  initial 
development,  execution,  evaluation,  and  subseguent  refinement  of 
this  STEP  validation  methodology  [7,9].  It  is  anticipated  that 
this  methodology  will  form  the  basis  for  validation  of  future  STEP 
Application  Protocols  (AP)  [5]. 

This  document  defines  the  Test  Plan  that  will  govern  testing  of  the 
first  CDIM  developed  by  the  PDES,  Inc.  Sheet  Metal  Project  (i.e., 
CDIM  SMI).  This  Test  Plan  specifies  the  test  objectives, 
methodology,  constraints,  requirements,  evaluation  criteria, 
resources,  deliverables,  and  issue  procedures  for  this  testing 
activity.  In  addition,  this  document  provides  a brief  introductory 
overview  of  CDIM  SMI  for  those  readers  unfamiliar  with  this  Sheet 
Metal  Project  effort. 
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2 . 0 Background  - CDIM  SMI  Overview 

Due  to  the  interest  of  PDES , Inc.  member  companies,  the  PDES , Inc. 
Sheet  Metal  Project  was  initiated  during  the  first  half  of  1990. 
This  project  has  focused  on  the  creation  of  an  international 
standard  for  the  exchange  and  sharing  of  product  data  pertaining  to 
design  and  manufacture  of  sheet  metal  parts  and  related  tooling 
[14].  The  first  major  effort  of  this  project  is  named  Context- 
Driven  Integrated  Model  (CDIM)  SMI.  Full-scale  development  of  CDIM 
SMI  began  in  September  1990,  with  participation  from  General  Motors 
(GM) /Electronic  Data  Systems  (EDS),  Boeing,  National  Institute  of 
Standards  and  Technology  (NIST) , Product  Data  Integration 
Technologies,  Inc.  (P.D.I.T.),  and  South  Carolina  Research 
Authority  (SCRA). 

2 . 1 CDIM  Purpose 

The  purpose  of  CDIM  SMI  is  to  test  and  validate  the  portions  of  the 
STEP  resource  models  that  are  within  the  scope  of  the  defined  CDIM 
application  context.  Within  the  CDIM  SMI  effort,  the  Sheet  Metal 
Project  will  produce  a CDIM  data  model,  testing  criteria,  and 
documented  test  results.  These  items  will  be  used  to  further  the 
development  of  STEP,  as  well  as  to  provide  much  of  the  needed 
information  for  one  or  more  sheet-metal-focused  STEP  Application 
Protocols  (AP)  [6]. 

2 . 2 CDIM  Scope 

The  scope  for  CDIM  SMI  is  specifically  defined  as  follows: 

CDIM  SMI  will  focus  on  standardization  of  the  product  data  for 
a shared  data  environment  to  support  the  design  of  dies/blocks 
for  sheet  metal  parts.  This  includes  the  interrelationships 
between  the  dies/blocks  and  the  parts  they  fabricate.  The 
CDIM  will  utilize  existing  STEP  resource  models.  [16] 

This  scope  includes: 

• Exchange  of  product  definition  data  as  it  relates  to  die 
design,  from  receipt  of  a die  request  to  the  providing  of 
corresponding  die  manufacturing  data. 

• Management  of  design  change  control  data  within  the  die 
design  process. 

• A focus  on  a shared  database  environment  to  reflect  internal 
PDES,  Inc.  member  company  needs  for  product  design  and 
manufacturing  automation  support. 

Candidate  STEP  resource  models  to  satisfy  data  requirements  for 
this  application  context  include  Geometry,  Topology,  Tolerance, 
Product  Structure  Configuration  Management  (PSCM),  Shape 


2 


Representation,  Form  Features,  Material,  Manufacturing,  and  the 
resource  model  subsets  that  exist  in  PDFS,  Inc.  CDIMs  A1 , A2 , and 
B3/B4  [8,9,10,12,17].  In  addition,  CDIM  SMI  will  use  the  STEP 
Integration  Framework  models  (i.e..  Generic  Product  Data  Model 
(GPDM)  and  Generic  Enterprise  Data  Model  (GEDM))  [1,2]. 

2 . 3 CDIM  Activities 

Before  initiation  of  this  Test  Plan  document,  the  following  CDIM 
SMI  activities  were  completed: 

• Development  of  an  activity  model  for  the  process  of  "Prepare 
for  Sheet  Metal  Part  Production"  [15].  This  model  is  roughly 
equivalent  to  the  Application  Activity  Model  (AAM)  of  a STEP 
Application  Protocol  [6]. 

• Development  of  a data  planning  model  for  the  in-scope  data 
as  identified  through  the  activity  modeling  process  [15]. 
This  model  is  roughly  equivalent  to  the  Application  Resource 
Model  (ARM)  of  a STEP  Application  Protocol  [6]. 

Both  of  these  models  were  reviewed  extensively  by  sheet  metal  die 
design  application  experts  from  within  PDFS,  Inc.  and  other 
automotive  and  aerospace  companies . 

At  the  conclusion  of  the  above  activities,  the  CDIM  SMI  project 
team  then  split  into  two  parallel  efforts:  the  Model  Team,  to 
develop  the  CDIM  data  model  from  existing  STEP  resource  models,  and 
the  Test  Team,  to  develop  the  test  strategy  as  documented  in  this 
Test  Plan,  develop  test  suites,  test  scenarios,  and  test  cases, 
obtain  test  data  from  industry  sheet  metal  die  design  application 
experts,  and  perform  the  CDIM  testing.  The  remainder  of  this 
document  is  concerned  strictly  with  CDIM  SMI  Test  Team  activities. 
For  further  information  on  other  CDIM  SMI  activities,  the  reader  is 
directed  to  the  CDIM  SMI  Project  Plan  [16]. 
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3 . 0 Test  Purpose 

This  section  outlines  the  scope  and  objectives  of  the  CDIM  testing 
process.  The  testing  purpose  and  the  CDIM  purpose  (as  given  in 
Section  2.1)  are  essentially  eguivalent.  The  testing  activity 
provides  the  basis  for  evaluating  the  validity  of  the  STEP  resource 
models  within  the  CDIM  scope  from  an  application  viewpoint. 
Development  of  the  CDIM  model  provides  the  viewpoint  or  context 
from  which  to  test  the  STEP  resource  models.  It  is  important  to 
note  that  evaluation  of  the  STEP  resource  models  cannot  occur 
without  some  context  on  which  to  base  data  reguirements . 

3 . 1 Test  Scope 

The  scope  of  the  CDIM  testing  will  include  those  activities  defined 
as  being  in-scope  by  the  CDIM  SMI  Activity  Model  and  the  data 
elements  defined  by  both  the  in-scope  Inputs,  Controls,  Outputs, 
and  Mechanisms  (ICOMs)  from  this  activity  model  and  the  CDIM  SMI 
Data  Planning  Model  [15]. 

All  attempts  will  be  made  to  provide  maximum  test  coverage  of  the 
CDIM  data  model  within  the  constraints  placed  upon  the  testing  team 
(see  Section  5.0,  Test  Constraints).  In  a practical  sense, 
sufficient  test  cases  cannot  be  obtained  or  developed  to  completely 
test  all  possible  aspects  of  the  sheet  metal  die  design  process 
from  every  company's  viewpoint.  The  task  that  is  possible, 
however,  is  to  test  from  several  viewpoints  using  sheet  metal  die 
design  application  experts  from  different  companies  and  industries 
to  obtain  a representative  cross-section  of  sheet  metal  die  design 
data  requirements.  This  approach,  along  with  a coordinated 
selection  of  test  suite  subject  areas,  will  ensure  that  the  test 
team  will  achieve  maximum  coverage  of  the  CDIM  data  model. 

3 . 2 Test  Objectives 

The  CDIM  SMI  Test  Team  has  identified  several  objectives  for  the 
CDIM  SMI  testing  activity.  These  objectives  are  provided  below. 

• Provide  data  to  further  STEP  resource  model  validation. 

• Identify  deficiencies  in  the  STEP  resource  models  through 
analysis  of  the  CDIM  SMI  data  model. 

• Determine  how  well  the  CDIM  SMI  model  satisfies  member 
company/industry  data  requirements  ( in  the  area  of  sheet  metal 
die  design) . 

• Identify  areas  of  the  CDIM  SMI  model  that  have  no 
corresponding  industry  data. 
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4 . 0 Test  Methodology 

This  section  provides  an  overview  of  CDIM  testing  methodology, 
descriptions  of  the  software  tools  used  in  the  testing  activity, 
and  CDIM  SMI  Test  Team  decisions  regarding  the  logistics  of  testing 
and  version  control  of  test  data. 

4.1  Elements  of  CDIM  Testing 

There  are  six  primary  elements  involved  in  CDIM  testing.  These 
elements  are  described  below  in  roughly  the  order  in  which  they  are 
performed . 

1)  Map  Data.  This  activity  is  largely  a manual  operation 
which  consists  of  mapping  test  case  data  to  entities  and 
attributes  within  the  CDIM  model.  This  mapping  operation  can 
be  performed  using  either  the  IDEFIX  wall  chart  or  the  Express 
version  of  the  data  model.  A two-step  mapping  approach  is 
used  as  described  below  to  enable  testers  to  validate  high- 
level  entity  relationships  before  mapping  detailed  attribute- 
level  test  data. 

Step  1 - High  Level  Mapping  - Mapping  is  performed  at  the 
entity  level.  Basic  entity  relationships  are  analyzed 
with  the  primary  question  to  be  answered  during  this  step 
being,  "Can  I generally  find  entities  within  the  CDIM 
model  which  correspond  to  my  test  case  data,  particularly 
for  key  test  case  elements?" 

This  step  concludes  with  a review  of  the  mapping  results 
by  both  the  testing  and  modeling  teams  to  validate  the 
relationships  of  the  test  data  to  the  entity  definitions. 
The  modeling  team  will  comment  on  the  mappings  and  the 
test  team  will  compare  individual  test  mappings  to 
provide  for  consistent  and/or  compatible  interpretations 
across  different  company  scenarios. 

Step  2 - Detailed  Mapping  - Mapping  is  performed  at  the 
attribute  level.  This  step  determines  if  the  CDIM  model 
satisfies  very  detailed  attribute  data  requirements 
(e.g.,  types,  number,  definitions,  groupings,  dependency) 
and/or  detailed  entity  data  requirements  (e.g., 
cardinality  relationships). 

A common  technique  used  during  this  step  is  called 
"instancing".  Instancing  consists  of  inserting  test  case 
data  into  tables,  database  schemas,  or  the  QDES  software 
tool  [11]  in  order  to  visually  analyze  the  relationships 
between  data  elements.  (Note:  QDES,  or  Quick  and  Dirty 
Editor  for  STEP,  is  described  in  Section  4.2  of  this 
document.)  This  technique  helps  verify  that  test  case 
data  is  properly  represented  in  the  CDIM  data  model. 
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This  step  also  concludes  with  a review  of  each  tester's 
mapping  results  by  both  the  testing  and  modeling  teams. 

Past  PDES , Inc.  CDIM  efforts  have  shown  that  the  mapping 
exercise  is  a significant  portion  of  the  overall  testing  task. 
This  detailed  examination  of  the  CDIM  documentation  typically 
identifies  several  issues  against  the  model's  ability  to 
represent  the  test  case  data.  Each  tester  will  also  interpret 
the  entity  and  attribute  usages  based  upon  the  CDIM  document. 
A comparison  of  each  tester's  interpretations  will  provide  the 
basis  for  evaluating  the  clarity  of  the  CDIM  documentation. 

2)  Analyze  CDIM  Model  Coverage.  After  completion  of  the 
mapping  task,  an  analysis  of  the  CDIM  data  model  coverage  is 
performed.  This  activity  determines  the  extent  of  the  model 
which  is  covered  by  available  test  data.  This  analysis 
assists  in  determining  which  entities  will  be  populated  into 
a testing  database  for  further  test  execution.  Entities  in 
the  CDIM  which  are  not  covered  by  test  data  need  not  be 
populated.  In  these  cases,  a test  issue  report  will  be 
written  and  a determination  of  the  entity's  status  (i.e., 
remove  from  model,  leave  in  model,  search  for  additional  test 
data,  etc.)  will  be  made  based  on  input  from  the  testing  team, 
modeling  team,  and  sheet  metal  die  design  application  experts. 

3)  Implement  the  Database.  This  activity  consists  of  creating 
a database  schema  from  the  CDIM  model  using  STEP  testing 
software  tools  (Note:  These  software  tools  are  described  in 
section  4.2  of  this  document).  This  operation  alone  is  guite 
valuable  in  that  it  locates  syntax  and  some  logic  errors 
within  the  CDIM  model. 

4)  Populate  Schema  with  Data.  This  activity  consists  of 
loading  test  case  data  into  the  database  schema  that  was 
created  in  the  previous  "Implement  the  Database"  activity. 
Test  data  can  be  loaded  into  the  database  through  a number  of 
methods.  The  most  direct  method  of  database  population  is  by 
writing  Structured  Query  Language  (SQL)  "Insert"  commands  to 
place  data  elements  into  the  database  tables.  Alternatively, 
a STEP  file  to  Oracle  database  load  program  (i.e.,  STEP 
Working  Form  to  SQL,  or  STEPwf-SQL)  [11],  which  is  discussed 
in  Section  4.2  of  this  document,  can  also  be  used  to  populate 
the  database.  The  specific  method  used  to  perform  this 
activity  is  typically  dependent  upon  the  format  of  the  given 
test  case  data.  If  the  data  is  in  the  form  of  a STEP  physical 
file,  the  most  obvious  choice  would  be  to  use  the  STEPwf-SQL 
software  tool  to  populate  the  database. 

5)  Write  and  Execute  Database  Queries.  This  activity  consists 
of  writing  and  executing  SQL  database  queries  to  retrieve 
desired  information  from  the  database  schema  populated  with 
test  case  data.  The  queries  ask  sample  real-world  questions 
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which  an  enterprise  must  be  able  to  answer  to  implement 
internal  systems.  Successful  execution  of  these  queries 
proves  that  an  access  path  to  those  data  elements  and  the 
necessary  relationships  between  data  elements  are,  in  fact, 
supported  by  the  CDIM  model . 

Query  execution  will  also  be  used  to  perform  interoperability 
and  "shared  data"  testing.  Based  upon  the  CDIM  SMI  focus  of 
a shared  database  environment,  test  case  data  from  all  testers 
will  be  populated  into  a single  database  instance.  Queries 
written  by  one  tester  for  specific  populations  of  data  will  be 
executed  against  populations  of  data  developed  by  other 
testers.  This  testing  activity  will  verify  consistency, 
usability,  and  data  sharing  aspects  of  the  CDIM  data  model. 

An  additional  technique  that  uses  SQL  queries  to  assess  the 
usability  of  the  data  model  will  also  be  performed.  This 
technique  uses  the  SQL  "Update"  facility  to  modify  information 
in  the  database  based  upon  the  content  of  previously  retrieved 
data.  This  "Update"  exercise  demonstrates  that  the  data  is 
usable  from  reasonable  business  perspectives. 

6)  Evaluate  and  Report  Results.  As  each  test  result  is 
obtained,  the  tester  must  make  a comparison  of  this  actual 
result  with  the  expected  result.  The  test  result  will  be 
recorded,  regardless  of  the  outcome  of  the  test.  If  the 
actual  result  does  not  match  the  expected  result,  a test  issue 
report  will  also  be  written.  This  test  result  documentation 
is  reported  to  the  CDIM  Model  Team  and  is  provided  as  one  of 
the  final  deliverables  from  the  CDIM  Test  Team. 

Current  plans  include  three  separate  test/fix/test  cycles  to 
validate  the  CDIM  SMI  data  model.  This  means  that  three  versions 
of  the  CDIM  model  will  be  tested  by  the  test  team.  After  each  test 
iteration,  all  test  results  and  test  issue  reports  will  be  provided 
to  the  model  team  to  incorporate  changes  into  the  next  release  of 
the  CDIM  model. 

4.2  Express-Based  Testing  Software  Tools 

The  software  tools  available  to  facilitate  the  testing  process  as 
outlined  above  are  based  upon  the  Express  language  representation 
of  a CDIM  model.  These  software  tools,  which  were  jointly 
developed  by  NIST  and  PDES,  Inc.,  have  been  used  in  past  PDES,  Inc. 
CDIM  projects.  An  overview  of  the  software  tools  to  be  used  in 
CDIM  SMI  testing  is  provided  below  [5,11]. 

1)  A set  of  software  tools  exists  to  perform  the  function  of 
implementing  the  database.  The  Fed-X  tool  performs  an  initial 
syntax  check  of  the  CDIM  Express  model.  The  Fed-X-SQL  tool 
generates  SQL  command  statements  (i.e.,  "Create"  statements) 
from  the  CDIM  Express  model  to  form  the  database  schema. 
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Database  load  utilities  are  then  used  to  initialize  the 
database  from  these  "Create"  statements. 

2)  Another  set  of  software  tools  exists  to  load  test  case  data 
into  the  database  schema.  This  test  data  must  be  in  STEP 
physical  file  format.  In  most  cases,  test  data  must  be  pieced 
together  because  few  STEP  translators  currently  exist  that  can 
provide  all  information  necessary  for  testing.  A common  route 
is  to  obtain  geometry  data  from  existing  Computer  Aided  Design 
(CAD)  systems  with  Initial  Graphics  Exchange  Specification 
(IGES)  translators  and  to  then  use  the  IGES  to  PDES  (ITOP) 
translator  test  tool  to  form  STEP  physical  files.  Once  in 
STEP  physical  file  format,  the  test  data  can  be  loaded  into 
the  database  schema  using  a STEP  Working  Form  to  SQL  database 
loader  software  tool  (i.e.,  STEPwf-SQL) . 

3)  Due  to  the  limitations  of  IGES,  existing  IGES  translators, 
and  the  IGES  to  PDES  translator,  all  test  case  data  cannot  be 
obtained  from  IGES  files.  A set  of  test  tools  addresses  the 
need  to  interactively  create  or  modify  test  case  data.  The 
Quick  and  Dirty  Editor  for  STEP  (QDES)  tool  is  used  to  create, 
modify,  or  otherwise  manipulate  test  case  data.  Items  such  as 
tolerance,  material,  or  product  structure  are  typically  added 
using  QDES  to  form  a complete  test  case.  The  Fed-X-QDES  tool 
is  used  to  establish  an  image  of  the  Express  model  necessary 
to  execute  QDES.  Also,  the  STEPparse-QDES  tool  is  used  to 
load  existing  STEP  physical  file  data  into  the  QDES  editor. 
The  output  of  QDES  is  once  again  a STEP  physical  file  to  be 
loaded  into  the  database  using  the  STEPwf-SQL  tool. 

4)  The  software  tools  available  to  aid  execution  of  SQL 
gueries  consist  strictly  of  database  utilities.  No  tools 
currently  exist  to  aid  evaluation  of  the  query  results,  which 
may  consist  of  an  "answer"  to  the  query  and/or  database 
software  error/warning  messages. 

5)  Other  STEP  validation  software  tools  exist  that  may  also 
prove  valuable  during  the  testing  process.  The  STEPparse-STEP 
software  tool  is  used  to  verify  the  syntax  of  a STEP  physical 
file  against  the  structure  of  an  Express  model.  The 
Visualizer  software  is  used  to  graphically  display  a STEP 
physical  file  and  to  test  the  validity  of  geometry  entities. 
Old  to  New  (OTON)  is  a tool  that  updates  a geometry  schema 
from  the  PDES,  Inc.  "GEOMIB"  representation  to  the  ISO  "pre- 
St.  Louis"  geometry  version.  The  Renumber  software  is  a tool 
that  renumbers  the  entity  identifiers  within  a STEP  physical 
file  so  that  test  cases  can  be  formed  by  appending  two  STEP 
files  together.  GetSTEP  locates  the  "parent"  or  "child" 
entities  of  a specified  entity  in  a STEP  physical  file. 
(Note:  This  paragraph  provides  a sampling  of  the 
miscellaneous  software  tools  available  to  the  tester  and  is 
not  intended  to  be  all-inclusive.) 
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4 . 3 B-Rep  and  Form  Feature  Testing  Strategy 

Two  separate  methods  will  be  used  to  test  both  the  Boundary 
Representation  (B-Rep)  solid  geometry  and  Form  Feature  portions  of 
the  CDIM  model.  Each  of  these  two  methods  validates  a different 
aspect  of  the  data  content.  Validation  of  the  B-Rep  and  Form 
Feature  constructs  within  the  CDIM  model  (and  also  to  some  extent, 
the  geometry  and  topology  constructs)  must  use  somewhat  different 
techniques  due  to  the  basic  focus  of  the  underlying  STEP  resource 
models.  Testing  of  other  portions  of  the  CDIM  is  done  from  what  is 
known  as  the  "external"  view,  or  user  view,  of  the  data.  The  CDIM 
SMI  Test  Team  believes  that  testing  of  the  B-Rep  and  Form  Feature 
constructs  within  the  CDIM  will  be  more  meaningful  if  done  from  an 
"internal"  view,  or  application  system  view.  For  this  reason,  the 
following  test  strategy  is  proposed. 

The  first  test  method  consists  of  comparing  the  internal  schemas  of 
existing  B-Rep/Form  Feature  system  implementations  with  the  STEP 
representation.  As  the  STEP  resource  models  should  support  any 
reasonable  mapping  from  any  system  implementation,  this  technique 
is  used  to  validate  the  correctness  of  the  STEP  representation.  In 
effect,  this  activity  performs  a rough  "requirements  analysis" 
phase  of  a STEP  translator  development  and  also  evaluates  the 
potential  success  of  an  exchange  of  B-Rep/Form  Feature  STEP  data 
between  modeling  systems.  It  is  recognized,  however,  that  this 
method  has  a high  probability  of  failure  due  to  inability  to  obtain 
the  internal  schemas  of  existing  modeling  system  implementations. 
In  most  cases,  this  information  is  considered  to  be  of  a 
proprietary  nature.  If  these  schemas  cannot  be  obtained  for  at 
least  two  existing  systems,  this  test  method  may  not  be  performed. 

The  second  test  method  will  validate  the  relationship  of  the  B-Rep 
and  Form  Feature  entities  to  other  parts  of  the  CDIM.  This  method 
is  performed  using  a similar  test  methodology  as  used  for  the  rest 
of  the  CDIM  (i.e.,  obtain  test  case  data,  perform  mapping  of  data 
to  the  CDIM  model,  load  data  into  database,  execute  SQL  queries, 
etc.).  As  described  below,  however,  this  testing  will  require  use 
of  an  additional  software  tool  to  generate  B-Rep  STEP  test  data. 
Unfortunately,  automated  tools  are  not  available  to  generate  Form 
Feature  STEP  test  data.  Thus,  Form  Feature  testing  by  this  method 
will  most  likely  require  creation  of  the  data  using  QDES  or  hand- 
population  of  the  database  using  the  SQL  Insert  utility. 

Initial  geometry  for  testing  B-Rep  will  be  obtained  from  solid 
models  of  appropriate  test  objects  (i.e.,  sheet  metal  parts  and/or 
dies)  from  solid  modeling  systems  available  within  GM/EDS  and 
Boeing.  In  order  to  generate  the  desired  B-Rep  STEP  test  data, 
this  data  must  then  be  transferred  to  a Parasolid  modeling  system. 
(Note:  A Parasolid  system  at  NIST  may  be  used  during  this  phase  if 
one  is  not  available  at  GM/EDS.)  This  transfer  may  consist  of 
creating  solid  models  of  appropriate  test  objects  within  Parasolid 
or  importing  the  data  to  Parasolid  from  the  other  modeling  systems. 
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The  feasibility  of  importing  data  to  Parasolid  from  internal  member 
company  systems  where  sheet  metal  part  and  die  designs  are  actually 
located  will  be  evaluated  by  the  CDIM  SMI  Test  Team. 

Once  the  appropriate  Parasolid  solid  model  is  created,  a software 
tool  developed  at  NIST  called  the  Parasolid  to  STEP  translator  [4] 
will  be  used  to  create  STEP  physical  files  which  contain  the  B-Rep 
data.  This  STEP  physical  file  can  then  be  used  in  the  testing 
process,  just  as  any  other  STEP  file  test  case  data. 

This  Parasolid-based  tool  was  selected  as  the  only  current 
practical  source  of  B-Rep  test  data  because,  although  other 
modeling  systems  are  available  at  member  companies,  this  process 
provides  test  data  in  a format  that  is  readily  usable  in  the 
existing  test  methodology. 

The  Parasolid  to  STEP  translator  has  some  limitations  that  must  be 
addressed  by  the  test  team.  These  limitations  are  outlined  below. 

• The  only  surface  types  currently  supported  by  this  tool  are 
the  plane,  sphere,  cone,  cylinder,  and  torus  (i.e.,  no 
sculptured  surfaces). 

• All  surface  edges  must  be  represented  by  a line,  circle,  or 
ellipse  (or  portions  of  these  entity  types). 

• The  STEP  physical  file  format  output  by  this  tool  was 
written  to  comply  with  a previous  version  of  the  STEP  standard 
(i.e.,  the  "Tokyo  version").  Software  support  may  be  required 
from  PDES,  Inc.  to  update  this  STEP  physical  file  to  comply 
with  the  latest  version  of  the  STEP  standard. 

Validation  of  the  B-Rep  aspects  of  the  CDIM  model  through  this 
method  will  also  require  that  the  tester  be  familiar  with  the 
concepts  of  solid  geometry  product  representation  and  be  able  to 
compare  and  critique  the  mapping  between  the  test  case  product 
model  and  the  resulting  STEP  file  and/or  database  schema. 

4 . 4 IDEF-Based  Testing 

An  original  objective  of  the  Sheet  Metal  Project  was  to  evaluate 
portions  of  the  CDIM  SMI  model  using  a testing  methodology  based 
upon  the  IDEFIX  representation.  This  test  methodology  had  not  been 
used  in  previous  PDES,  Inc.  CDIM  projects  and,  as  such,  software 
tools  did  not  exist  to  facilitate  this  testing.  General  software 
tool  requirements  were  developed  to  illustrate  the  feasibility  of 
this  testing  approach.  These  requirements  were  recorded  in  the 
Sheet  Metal  Project  software  tools  requirements  document  [13]. 

The  focus  of  CDIM  SMI  testing  has  been  modified  somewhat  since  this 
initial  plan.  Primary  CDIM  testing  activities  will  be  done  using 
the  Express-based  approach,  as  previously  described.  Some 
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"investigative"  IDEF-based  testing  of  CDIM  SMI  is  still  planned, 
however,  during  the  later  stages  of  testing,  most  likely  during  the 
third  testing  iteration.  The  purpose  for  this  testing  will  be  to 
verify  that  test  results  obtained  previously  from  the  Express  model 
can  be  duplicated  when  analyzing  the  IDEFIX  model.  Results  of  this 
testing  would  provide  tangible  evidence  (which  has  previously  been 
nonexistent)  to  validate  the  CDIM  model  conversion  process  from 
IDEFIX  to  Express.  This  evidence  would  also  reflect  upon  the 
conversion  process  of  the  STEP  resource  models. 

The  IDEF-based  testing  concept  is  basically  the  same  as  that  used 
in  past  CDIM  testing,  except  that  the  database  schema  is  derived 
from  the  IDEFIX  representation,  rather  than  Express  representation. 
The  concepts  of  creating  the  database  schema,  obtaining  or  editing 
test  case  data,  loading  the  test  case  data  into  the  database 
schema,  and  performing  database  gueries  to  extract  desired 
information  are  all  still  valid.  As  defined  in  the  tools 
reguirements  document,  new  or  modified  software  tools  to  support 
these  activities  would  be  beneficial. 

4.5  Logistics  of  CDIM  SMI  Testing 

Five  options  have  been  identified  concerning  the  logistics  of  CDIM 
SMI  testing.  Each  option  describes  a method  for  accessing  the  NIST 
National  PDES  Testbed  (NPT)  in  Gaithersburg,  MD,  and/or  the  SCRA 
PDES  Development  Lab  (PDL)  in  Charleston,  SC.  The  goal  of  the  CDIM 
SMI  Test  Team  is  to  minimize  travel  and  to  maximize  use  of  remote 
computer  access  to  perform  testing  activities.  The  five  options 
are  listed  below,  with  specific  considerations  listed  in 
parenthesis  after  each  item. 

1)  X-Windows  terminal  (X-terminal)  remote  access  (9600  baud 
connection  available  through  modems  supplied  by  SCRA,  QDES  and 
Visualizer  software  not  yet  available  through  this  connection, 
X-terminal  hardware  must  be  obtained,  some  travel  required) 

2)  Internet  network  access  (available  at  NIST  and  SCRA,  need 
Tl-1.6Mb  capability,  some  tester  sites  do  not  have  Internet) 

3)  Personal  Computer  (PC)  remote  access  (9600  baud  connection 
available  through  modems  supplied  by  SCRA,  some  window 
emulation  available  with  PC-EMACS  software  tool,  QDES  and 
Visualizer  software  not  available,  some  travel  required) 

4)  Local  access  of  QDES  and  Visualizer  software,  PC  remote 
access  for  other  tools  (limited  travel,  need  software 
installed  at  tester  companies) 

5)  All  testing  done  locally  at  NIST  or  SCRA  Testbeds 
(extensive  travel) 

The  CDIM  SMI  Test  Team  decision  is  to  pursue  Option  4 above  as  the 
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top  priority,  with  Option  3,  and  then  Option  5 as  fail-back 
positions.  Options  3 and  5 are  actually  status  quo  and  are  being 
used  by  other  PDES,  Inc.  CDIM  projects. 

The  X-terminal  option  did  not  provide  enough  functionality  increase 
to  warrant  selection  over  a PC.  The  CDIM  SMI  Test  Team  has 
documented  and  distributed  its  evaluation  of  the  X-terminal 
connection  capability.  The  CDIM  SMI  Team  also  plans  to  continue  to 
monitor  progress  on  establishing  X-terminal  connection  to  the 
testbeds . 

The  Internet  option  could  not  be  selected  due  to  the  lack  of 
Internet  connection  capability  at  member  company  sites. 

The  CDIM  SMI  Test  Team  will  gather  information  to  assess  the 
feasibility  and  impact  of  installing  the  QDES  and  Visualizer  tools 
locally  at  each  tester's  member  company  site.  Hardware  and 
software  requirements/availability  and  required  implementation 
schedule  will  also  be  determined. 

If  the  local  access  option  proves  feasible,  one  suggestion  that  has 
been  proposed  is  that  during  the  testing  process,  the  QDES  image  of 
the  CDIM  schema  could  potentially  be  generated  at  one  facility  and 
shared  among  the  other  sites.  This  practice  would  save  many  hours 
for  each  tester  not  involved  in  generating  this  image. 

4.6  Revision  Control  System  (RCS) 

The  CDIM  SMI  Test  Team  will  investigate  use  of  the  Revision  Control 
System  (RCS)  to  manage  version  control  of  test  data.  RCS  is  a 
configuration  management  tool  that  is  available  at  both  the  NIST 
and  SCRA  testbeds  and  has  previously  been  used  to  manage  version 
control  of  documentation  [3].  Past  PDES,  Inc.  CDIM  testing  efforts 
have  found  version  control  of  test  data  to  be  one  of  the  largest 
problems  to  overcome.  Proper  versions  of  STEP  physical  file  test 
cases.  Express  files,  database  schemas,  SQL  gueries,  etc.  must  be 
used  together  to  ensure  that  any  given  test  yields  meaningful 
results.  The  CDIM  SMI  Test  Team  will  identify  the  test  data  types 
to  be  controlled,  procedures  to  be  established,  and  interaction 
between  potentially  multiple  testing  sites  after  gaining  more 
experience  in  the  testing  process  and  in  using  the  RCS  utility. 
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5 . 0 Test  Constraints 


Several  items  have  been  identified  as  constraints  to  the  CDIM 
testing  activity.  These  constraints  serve  to  limit  both  the 
quantity  and  quality  of  CDIM  test  results.  All  efforts  will  be 
made  to  overcome  obstacles  in  the  testing  process,  however,  and  to 
meet  all  test  objectives.  These  constraints  are  listed  below 
primarily  to  provide  explanation  for  decisions  made  in  this  test 
plan  regarding  test  methodology,  CDIM  model  coverage,  etc. 

• Complexity  of  Test  Data  - As  test  data  becomes  more  complex, 
the  task  of  selecting  appropriate  test  data  to  form  specific 
test  cases  becomes  more  difficult.  Some  of  the  available  die 
design-related  test  data  is  anticipated  to  be  quite  complex, 
especially  in  the  area  of  geometry  representation  (i.e., 
surfaces  and  B-rep  solids).  Although  a thorough  understanding 
of  the  mathematics  behind  these  representations  is  not 
actually  necessary,  the  tester  must  understand  the  test  data 
well  enough  to  compose  meaningful  tests  and  to  interpret  test 
results.  Similar  complex  test  data  situations  may  arise  due 
to  differences  in  terminology  between  companies  and  in  unique 
data  combinations,  such  as  in  internally-developed  forms  and 
documents . 

• Availability  of  Test  Data  - Test  data  must  be  obtained  from 
the  sheet  metal  die  design  application  experts  in  order  to 
test  the  accuracy  and  completeness  of  the  CDIM  model.  If 
appropriate  test  data  is  not  available  to  test  specific 
aspects  of  the  model,  these  portions  of  the  CDIM  will  not  be 
validated.  This  constraint  is  particularly  relevant  when 
testing  specific  "to-be"  aspects  of  the  die  design  process 
that  are  included  in  the  CDIM  scope  (as  opposed  to  the  "as-is" 
die  design  process). 

• Software  Tools  Limitations  - The  testing  process  will  be 
constrained  by  software  tools  in  terms  of : 

• Availability  - Does  a tool  exist  to  perform  a desired 
function  and  can  the  tester  access  it? 

• Capability  - How  well  or  complete  does  a given  tool 
perform  a desired  function? 

• Applicability  - Does  a given  tool  perform  the  actual 
function  needed  for  a specific  test  activity? 

These  software  tool  limitations  largely  determine  the  time 
required  to  perform  a testing  activity  (and  indirectly  the 
practicality  of  that  testing  activity) . If  appropriate 
software  tools  are  not  available  when  needed  to  perform 
specific  phases  of  the  testing  process,  CDIM  SMI  testing  will 
not  obtain  the  desired  results  in  the  allotted  time. 
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• Project  Schedule  - CDIM  SMI  testing  will  also  be  constrained 
by  the  amount  of  time  provided  in  the  project  schedule.  Test 
results  must  be  obtained,  documented,  and  presented  in  a 
timely  manner  in  order  for  project  management  to  continue 
support  and  funding  for  this  and  subsequent  CDIM  activities. 
As  the  PDES , Inc.  CDIM  methodology  is  quite  schedule-driven, 
testing  may  have  to  be  halted  before  total  completion  in  order 
to  meet  deliverable  due  dates  as  specified  in  the  project 
schedule.  The  CDIM  SMI  project  schedule  is  provided  for 
reference  in  Appendix  A. 

• Available  Resources  - CDIM  SMI  testing  will  be  performed 
using  available  resources,  specifically  those  human  resources 
assigned  to  the  CDIM  Test  Team.  Any  increase  in  human 
resources  (within  reason)  assigned  to  the  testing  activity 
would  increase  the  amount  of  CDIM  model  test  coverage  and 
decrease  the  time  required  to  accomplish  a set  of  testing 
tasks.  A decrease  in  human  resources  would  have  the  opposite 
effects . 
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6 . 0 Test  Requirements 

This  section  documents  the  expected  deliverables  from  the  CDIM 
Model  Team  and  the  methods  by  which  these  items  will  be  tested. 

6.1  Model  Team  Deliverables 

The  CDIM  SMI  Test  Team  expects  to  receive  the  following 
deliverables  from  the  CDIM  SMI  Model  Team.  Each  deliverable  will 
be  tested  using  the  methods  outlined  in  Section  6.2,  CDIM 
Deliverable  Evaluation  Methods. 

1)  Introduction/Overview  Document 

2)  CDIM  Express  model  - Copies  of  this  model  should  be 
provided  in  both  paper  and  electronic  form.  The  electronic 
copy  should  compile  error-free  using  the  Fed-X  software  tool. 

3)  Glossary/Definitions  - Definitions  of  the  data  model 
entities,  attributes,  terms,  relationships,  etc.  should  be 
provided  as  either  an  appended  glossary  or  imbedded  in 
appropriate  locations  throughout  the  body  of  the  CDIM 
document. 

4)  Data  Population  Examples  - The  purpose  for  these  examples 
is  to  demonstrate  entity  usage.  These  population  examples  can 
be  based  upon  either  the  Express  or  IDEFIX  models  and  can 
consist  of  the  population  of  tables  or  STEP  physical  files. 

5)  IDEFIX  model  - One  IDEFIX  wall  chart  should  be  provided  for 
each  tester  site.  The  IDEFIX  model  should  be  fully-attributed 
and  normalized  to  remove  many-to-many  relationships. 
Definitions  are  not  required  in  the  documentation  of  the 
IDEFIX  model,  except  to  explain  items  created  during  the 
normalization  process.  (Note:  definitions  of  the  other  items 
will  already  exist  in  the  Express  model  documentation.)  In 
addition,  an  electronic  copy  of  the  IDEFIX  Structural  Modeling 
Language  (SML)  format  should  be  provided  for  all  versions  of 
the  CDIM  model  except  for  VO . 1 . The  SML  is  used  as  input  to 
a software  tool  called  Leverage  and  will  be  used  during  the 
iDEF-based  testing  phase. 

In  addition,  the  CDIM  SMI  Model  Team  will  also  provide  a walk- 
through of  the  CDIM  model  delivered  to  the  test  team.  This  walk- 
through will  at  a minimum  include  presentations  on  the  CDIM  Express 
model  and  IDEFIX  wall  charts.  Additional  model  walk-throughs  may 
be  required  during  later  stages  of  the  testing  process  as  deemed 
necessary  by  the  test  team. 

Other  agreements  between  the  model  team  and  test  team  regarding 
delivery  of  the  CDIM  model  are  as  follows; 
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• The  model  team  will  select  specific  baseline  versions  of  the 
STEP  resource  models  and  Express  Specification.  The  test  team 
will  use  this  information  to  determine  the  impact  on  the 
testing  software  tools. 

• The  initial  release  of  the  CDIM  (VO.l)  will  not  contain 
entities  not  already  found  in  the  STEP  resource  models.  The 
model  team  will  instead  provide  a tentative  issue  to  the  test 
team  for  verification  using  the  test  data.  The  test  team  will 
validate  that  a gap  actually  exists  in  the  CDIM  data  model. 

6.2  CDIM  Deliverable  Evaluation  Methods 


The  following  are  methods  or  techniques  which  will  be  used  to  test 
the  CDIM  deliverables. 


• Build  Schema 

• Compare 

• Compile 

• Critique 

• Execute 

• Map 

• Populate 

• Query 

• Query  Path  Analysis 

• Update 


Build  a database  schema  from 
the  Express  model . 

Compare  the  IDEFIX  model  to  the 
Express  model . 

Compile  the  Express  model  using 
the  Fed-X  software  tool. 

Read,  analyze,  and  evaluate  the 
test  item. 

Perform  the  tasks  in  the  data 
population  example. 

Find  places  in  the  model  for 
the  test  data. 

Load  test  data  into  the 
database . 

Retrieve  data  from  the  database 
using  SQL  statements. 

Analyze  queries  that  retrieve 
the  same  or  similar  data. 
Modify  data  in  the  database 
using  SQL  statements. 


Each  part  of  the  CDIM  data  model  documentation  packet  will  be 
tested  using  the  techniques  outlined  below. 


1 ) Introduction/Overview  Document  - Critique 

2)  CDIM  Express  model  - Compile,  Build  Schema 

Characteristics  of  the  Express  model  to  be  evaluated: 

a)  Entities  - Critique,  Map,  Populate 

b)  Relationships  ~ Critique,  Map,  Populate 

Characteristics  of  relationships  to  be  evaluated: 

• Existence 

• Semantics 

• Cardinality 

• Dependence 
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c)  Attributes  - Critique,  Map,  Populate 

Characteristics  of  attributes  to  be  evaluated: 

• Existence 

- missing 

- extraneous 

- duplicate/redundant 

• Semantics 

- wrong  type 

- poorly  named 

• Key  vs . NonKey 

“ dependency 

- uniquely  identifying 

- adequate 

- minimally  sufficient 

d)  Keys  - Critique,  Map,  Populate 

(i.e..  Express  constraints) 

Characteristics  of  keys  to  be  evaluated: 

• Dependency 

• Uniquely  identifying 

- adequate 

- minimally  sufficient 

e)  Cardinalities  - Critique,  Map,  Populate,  Compare 

f)  Categorizations  - Critique 

g)  Nomenclature  - Critique 

h)  Functions  and  "derived"  information  - Critique 

i)  Presentation  - Critique 

j ) Redundancy  - Query  Path  Analysis 

k)  Usability  - Query,  Update 

l)  Rules  - Critique,  Map,  Populate 

3)  Glossary/Definitions  - Critique 

4)  Data  Population  Examples  - Critique,  Execute 

5)  IDEFIX  model  - Critique 

Characteristics  of  the  IDEFIX  model  to  be  evaluated: 

a)  Entities  - Critique,  Map,  Populate 

b)  Relationships  - Critique,  Map,  Populate 

Characteristics  of  relationships  to  be  evaluated 

• Existence 

• Semantics 

• Cardinality 

• Dependence 

c)  Attributes  - Critique,  Map,  Populate 

Characteristics  of  attributes  to  be  evaluated: 

• Existence 

- missing 

- extraneous 

- duplicate/redundant 

• Semantics 

- wrong  type 
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- poorly  named 

• Key  vs.  NonKey 

- dependency 

- uniquely  identifying 

- adequate 

- minimally  sufficient 

d)  Keys  - Critique,  Map,  Populate 

(i.e.,  IDEFIX  attributes) 

Characteristics  of  keys  to  be  evaluated; 

• Dependency 

• Uniquely  identifying 

- adequate 

- minimally  sufficient 

e)  Cardinalities  - Critique,  Map,  Populate,  Compare 

f)  Categorizations  - Critique 

g)  Nomenclature  - Critique 

h)  Functions  and  "derived"  information  - Critique 

i)  Presentation  - Critique 

j)  Redundancy  - Query  Path  Analysis 

k)  Usability  - Query,  Update 
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7 . 0  Test  Procedure 


This  section  describes  the  terminology  of  the  testing  process, 
details  of  the  testing  procedure,  and  the  relationships  between 
each  component  of  a specific  test. 

7 . 1 Test  Suites 

Information  to  be  tested  is  formally  documented  in  test  suites.  A 
test  suite  is  a collection  of  context  diagrams,  with  corresponding 
test  scenarios  and  test  cases.  Test  data  is  also  reguired  to 
create  the  specific  test  cases.  The  test  suite  perspective  is 
company-specific  and  defines  that  company's  view  of  a particular 
area  of  interest.  The  areas  of  interest  are  selected  from  the 
lowest  level  activity  boxes  of  the  CDIM  SMI  Activity  Model . One 
test  suite  is  created  for  each  area  of  interest/company 
combination.  It  is  anticipated  that  each  tester  will  develop  the 
initial  structure  for  multiple  test  suites  through  interaction  with 
the  sheet  metal  die  design  Application  Experts  (AEs).  Each  tester 
will  actually  focus  on  selected  test  suites,  however,  with  the 
subsequent  test  activities  of  remaining  test  suites  to  be  performed 
as  project  resources  and  schedule  allow.  Appendix  C,  Test  Suite 
Example,  further  illustrates  the  appearance  of  a test  suite  and  the 
relationships  between  test  suites,  test  scenarios,  test  cases,  and 
test  data. 

7.1.1  Test  Suite  Context  Diagrams 

Test  suites  contain  context  diagrams  to  define  detailed  areas  of 
interest  within  a particular  subject  (i.e.,  the  subject  domain  for 
a set  of  testing  activities).  Context  diagrams  are  equivalent  to 
an  IDEFO  activity  model  with  only  one  activity.  Just  as  in  IDEFO 
activity  modeling,  the  name  of  the  activity  is  located  within  a box 
and  all  Inputs,  Controls,  Outputs,  and  Mechanisms  (ICOMs)  are  shown 
around  the  outside  of  the  box.  An  example  context  diagram  is  also 
shown  in  Appendix  C. 

The  context  diagrams  are  developed  from  a detailed  decomposition  of 
selected  portions  of  the  overall  CDIM  SMI  Activity  Model.  This 
decomposition  of  the  CDIM  SMI  Activity  Model  occurs  during 
workshops  held  with  sheet  metal  die  designers  from  various 
companies.  Since  a goal  of  CDIM  SMI  testing  is  to  provide  100% 
test  coverage  across  the  CDIM  model , high-level  context  diagram 
subject  areas  should  optimally  be  selected  previous  to  these  AE 
workshops  to  provide  maximum  test  coverage. 

The  context  diagrams  are  generic  in  that  they  may  apply  to  any 
company  involved  in  developing  the  diagrams.  Each  context  diagram 
may  appear  only  once  in  a given  test  suite.  The  diagrams  may  be 
used  in  multiple  test  suites,  however,  if  multiple  companies 
perform  the  same  activities. 
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7.1.2  Test  Scenarios 


Test  scenarios  provide  a textual  description  of  specific  aspects  to 
be  tested  within  a given  test  suite  subject  area.  Each  test 
scenario  consists  of  a single  context  diagram  with  company-specific 
definitions  for  the  activity  and  iCOMs.  These  definitions  provide 
the  company's  interpretation  of  the  context  diagram.  If  an  ICOM  on 
the  context  diagram  is  not  used  by  a particular  company,  that  ICOM 
is  not  included  in  the  test  scenario. 

Each  test  suite  must  contain  one  or  more  test  scenarios.  Within  a 
given  test  suite,  however,  a context  diagram  can  only  be  used  in 
one  test  scenario. 

7.1.3  Test  Data 

Test  data  consists  of  all  real-world  data  available  to  a tester  to 
use  during  CDIM  SMI  testing.  This  test  data  will  be  provided  by 
Sheet  Metal  Project  member  companies  and  other  companies  whose 
activities  are  modeled  in  the  test  suite.  The  test  data  will  be 
the  actual  representation  of  the  data  used  within  the  particular 
company  (i.e.,  paper  drawings,  CAD  system  IGES  files,  process 
plans,  change  order  information,  specifications  and  other 
documents,  etc.).  The  CDIM  SMI  Test  Team  is  responsible  for 
collecting  appropriate  and  sufficient  test  data  to  support  testing 
of  the  desired  test  scenarios.  To  achieve  this  result,  the  test 
scenarios  should  be  used  as  a guideline  when  collecting  test  data 
from  the  Application  Experts. 

7.1.4  Test  Cases 

Test  cases  consist  of  the  portions  of  the  test  data  reguired  to 
substantiate  the  data  elements  of  a specific  test  scenario.  In 
other  words,  test  cases  map  the  test  data  to  the  ICOMs  in  the  test 
scenario  context  diagram.  The  test  scenarios  provide  the  context 
from  which  to  perform  testing  and  the  test  data  provides  the 
information  to  use  during  the  mapping  activity  and  to  load  into  the 
database.  The  test  data  used  within  the  test  cases  must  come  from 
the  same  company  for  which  the  test  suite  was  developed.  Also, 
each  test  scenario  must  contain  at  least  one  test  case. 

Each  test  case  also  includes  the  "guery  strategy"  to  be  used  for 
testing  the  data  elements  within  the  test  case.  Analysis  of  the 
SQL  gueries  and  their  results  is  used  to  evaluate  the  data 
dependencies  within  the  CDIM  data  model.  The  guery  strategy  used 
for  each  test  case  is  selected  from  one  of  the  three  options  below: 

• Write  pseudo-gueries  based  on  the  context  boxes  and  test 

data;  then  check  if  they  are  allowed  by  the  CDIM  model. 

• Write  gueries  based  on  the  structure  of  the  CDIM  model;  then 

check  if  they  match  the  context  boxes  and  test  data. 
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• Do  not  write  queries  for  this  test  case  and  state  the 
reasons  for  this  decision.  Possible  reasons  for  selecting 
this  option  include  a significant  number  of  common  data 
elements  among  several  test  cases  (i.e.,  to  minimize  redundant 
testing)  or  a lack  of  sufficient  test  data  to  analyze  the  path 
of  a meaningful  query. 

7 . 2 Test  Execution 

Execution  of  any  given  test  requires  initial  development  of  test 
suites,  test  scenarios,  and  test  cases.  The  test  data  must  be 
obtained  before  test  cases  can  be  formed.  While  executing  the 
tests,  the  Test  Issue  Log  format  must  be  available  to  record 
incorrect  or  unexpected  test  results. 

The  actual  steps  in  the  execution  of  a test  vary  depending  on  the 
test  method.  If  a high-level  test  is  performed  without  use  of 
software  tools,  or  "on  paper",  the  concept  of  test  execution  is 
difficult  to  document.  When  actual  database  queries  are  performed, 
however,  test  execution  steps  can  be  more  easily  defined.  These 
execution  steps  were  outlined  in  Section  4.0  of  this  document. 

When  problems  or  deficiencies  are  found  during  CDIM  SMI  testing,  a 
test  issue  report  is  generated  and  a decision  must  be  made 
regarding  subsequent  test  execution.  The  default  response  to  this 
situation  is  to  fully  document  the  issue  and  continue  testing. 
This  action  should  be  sufficient  for  the  majority  of  test  issues. 

Other  times,  however,  testing  cannot  continue  without  first 
correcting  the  problem.  Two  actions  can  occur  at  this  point: 

1)  the  tester  can  make  a quick  fix  on-the-fly  if  the  solution  is 
quite  trivial,  or  2)  testing  can  be  suspended  until  the  CDIM  Model 
Team  can  provide  the  proper  fix.  Both  of  these  actions  have 
serious  consequences.  The  first  option  should  be  avoided  if  at  all 
possible,  and  if  done,  the  fix  should  be  documented  extremely  well 
and  reverified  in  later  releases  of  the  CDIM  model.  From  a project 
schedule  standpoint,  the  second  option  should  also  be  avoided  and 
will  be  chosen  only  after  independent  verification  of  the  problem 
and  test  team  consensus  has  been  reached. 

7.3  Test  Evaluation  Criteria 

Test  evaluation  criteria  are  largely  determined  on  a case-by-case 
basis,  depending  on  the  characteristics  of  the  particular  test. 
Evaluation  of  test  results  is  also  quite  subjective  based  upon  the 
tester's  expected  results  and  interpretation  of  actual  test 
results.  The  test  results  obtained  by  this  process  do  not  actually 
guarantee  that  perceived  correct  results  are  indeed  satisfactory, 
but  only  that  specific  deficiencies  or  issues  have  been  identified. 
The  primary  evaluation  criteria  used  in  the  testing  process  is 
therefore  the  tester's  opinion  on  whether  the  actual  test  result 
matches  the  expected  test  result.  The  tester  is  expected  to 
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generate  a test  issue  report  for  any  mismatch  or  unexpected  result. 
As  all  test  issue  reports  are  reviewed  by  the  entire  test  team 
before  further  distribution,  some  assurance  is  provided  that  the 
issue  is  indeed  warranted. 
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8 . 0 Resources 


The  resources  required  for  testing  CDIM  SMI  are  quite  significant. 
This  section  provides  the  requirements  and/or  current  availability 
for  the  following  types  of  resources  necessary  for  the  testing 
process:  people,  facilities,  software  tools,  and  training. 

8 . 1 People 


The  CDIM  SMI  test  team  consists  of  the  following  personnel,  listed 
below  with  their  level  of  effort  and  their  company  affiliation: 


Kevin  Jurrens,  Team  Leader 
Gloria  Roth  Robinson 
Shyhdong  (Dong)  Tsuei 
Bill  Shaffer 
Jack  Skeels 
Frank  Demasek 


100%  NIST 
100%  EDS 
100%  EDS 
100%  Boeing 
30%  P.D.I.T. 
25%  EDS 


The  primary  test  team  consists  of  Kevin,  Gloria,  Dong,  and  Bill. 
This  core  team  will  participate  in  all  aspects  of  the  testing 
process.  It  is  recognized  that  the  size  of  this  core  test  team  is 
smaller  than  actually  desired  to  perform  thorough  testing  of  CDIM 
SMI.  Jack  will  be  primarily  involved  in  project  management 
activities,  application  expert  workshops,  test  team  training,  and 
testing  guidance.  Frank  will  participate  with  the  test  team  in 
validating  the  Solid  Geometry  Boundary  Representation  (B-rep)  and 
Form  Feature  aspects  of  the  CDIM  model.  In  addition,  it  is 
expected  that  as  the  CDIM  model  becomes  more  mature  (i.e.,  after 
about  the  second  iteration) , less  actual  modeling  work  will  be 
needed  and  some  members  of  the  CDIM  modeling  team  will  then 
participate  in  testing  activities. 

As  documented  in  the  CDIM  SMI  Project  Plan,  the  following 
assignments  have  been  made  within  the  test  team  to  distribute  task 
responsibilities : 


Schedules,  Agendas, 
Distribution,  and 
Announcements 

Scribe 

Scribe  Backup 

Activity  Model 
Administrator 

Issue  Log  Custodian 

Issue  Log  Backup 


Kevin  Jurrens 

Gloria  Roth  Robinson 
Kevin  Jurrens 
Gloria  Roth  Robinson 

Bill  Shaffer 
Dong  Tsuei 
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Reference  Library 


Gloria  Roth  Robinson 


Document  Editor 


Kevin  Jurrens 


Schema  Compiler 


Dong  Tsuei 


SQL  Language  Expert 


Dong  Tsuei 


Application  Expert  Liaison  Gloria  Roth  Robinson 

Bill  Shaffer 


Other  human  resources  will  be  provided  by  SCRA  and  the  Sheet  Metal 
Project  support  team  (i.e.,  selected  members  of  the  PDES,  Inc.  Core 
Program)  to  support  program  management  activities  and  project 
reviews,  respectively.  In  addition,  NIST  and  SCRA  PDES  Testbed 
personnel  will  provide  hardware/software  support  and  training  as 
necessary . 

8.2  Facilities 

Testing  of  CDIM  SMI  will  require  use  of  both  the  NIST  National  PDES 
Testbed  (NPT)  and  the  SCRA  PDES  Development  Lab  (PDL).  Currently, 
the  NIST  Testbed  has  been  designated  as  the  primary  testing 
facility  for  the  Sheet  Metal  Project.  This  decision  was  based  upon 
the  relative  use  of  each  of  the  testbeds.  Other  factors,  such  as 
testbed  readiness  and  software  availability,  are  quite  similar  at 
both  testbeds  and  do  not  affect  this  decision.  Distribution  of 
PDES,  Inc.  CDIM  testing  activities  is  currently  under  review, 
however,  and  this  selection  may  change  in  the  future. 

As  previously  mentioned  in  Section  4.5,  Logistics  of  CDIM  SMI 
Testing,  remote  access  to  the  testbeds  through  dial-in  modems  will 
facilitate  a large  portion  of  the  testing.  The  majority  of  the 
CDIM  testing  software  tools  are  not  graphics-based  and  will 
function  through  a simple  PC  connection.  As  remote  access 
capability  is  supported  at  both  testbeds,  test  team  member 
companies  will  use  local  hardware  and  software  as  required  to 
remotely  access  the  testbed  computers. 

When  a particular  testing  activity  cannot  be  performed  locally  at 
a tester's  site  or  through  the  testbed  remote  connection,  that 
activity  will  occur  at  either  NIST  or  SCRA  as  chosen  by  the  test 
team.  The  selected  testbed  must  be  reserved  in  advance  for  these 
activities  in  order  to  avoid  conflict  with  other  PDES,  Inc.  or  CDIM 
teams . 

Additional  facility  requirements  include  member  company  and  other 
PDES,  Inc.  conference  rooms  for  team  meetings.  Optimal  facility 
requirements  for  conducting  CDIM  development  and  testing  meetings 
are  identified  in  Appendix  H of  the  Testing  Criteria  Requirements 
Analysis  Project  (TC-RAP^  Final  Report  [ 7 ] . Although  some  items 
are  more  a matter  of  convenience  rather  than  necessity,  this 
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appendix  includes  items  such  as  an  overhead  projector,  paper  flip 
chart,  FAX  machine,  copy  machine,  secretarial  support,  outside 
telephone  line,  basic  office  supplies,  vending  machines,  PC  with 
printer,  hotel/restaurant/hospital  information,  evening  building 
access,  etc. 

8.3  Software  Tool  Requirements 

Software  tool  requirements  for  CDIM  SMI  were  originally  identified 
in  the  document.  Tools  Requirements  for  CDIM  SMI  Development  and 
Test  [13].  These  requirements  have  not  changed  in  their  basic 
content  since  this  original  document.  Some  of  the  tools,  such  as 
electronic  mail  or  ModelPro,  have  already  been  provided  and  are 
being  used  for  Sheet  Metal  Project  activities.  Other  tools,  such 
as  the  activity  modeling  tool,  were  not  available  when  needed  at 
specific  stages  of  the  project  and  team  members  had  to  use  less 
efficient  methods  to  complete  the  task.  The  tool  requirements  in 
this  document  should  still  be  considered  current  and  any  tool  not 
yet  provided  would  further  enhance  project  productivity  when 
available. 

Due  to  the  initial  project  focus  on  IDEF-based  testing,  the 
original  tools  requirement  document  did  not  address  the  need  for 
the  Express-based  NIST  and  PDES,  Inc.  STEP  testing  tools.  With  the 
change  in  CDIM  SMI  testing  methodology,  however,  these  software 
tools  are  now  of  primary  importance  to  the  test  team.  As  discussed 
in  Section  4 . 2 of  this  document,  this  set  of  software  tools  will  be 
used  in  its  entirety  and  should  be  available  at  both  NIST  and  SCRA 
Testbeds . 

The  following  software  tool  requirements  are  the  highest  priority 
new  capabilities  needed  by  the  CDIM  SMI  Test  Team.  These 
requirements  may  or  may  not  have  been  identified  in  the  original 
tools  requirements  document  and  are  provided  below  in  priority 
order.  These  software  tool  requirements  have  been  communicated  to 
the  PDES,  Inc.  software  tool  development  group. 

• In  accordance  with  the  test  team  decision  discussed  in 
Section  4.5  of  this  document  to  pursue  local  access  of  QDES 
and  the  Visualizer  software  at  each  tester's  member  company 
site,  assistance  and  support  will  be  expected  from  the  PDES, 
Inc.  tools  developers  to  establish  these  environments.  CDIM 
Test  Team  members  are  currently  gathering  information  to 
assess  the  feasibility  and  impact  of  this  test  environment. 

• Enhancements  are  required  to  the  IGES  to  PDES  (ITOP) 
translator  software  tool . This  tool  currently  supports  a 
limited  set  of  IGES  entities.  This  set  must  be  expanded  to 
accommodate  test  data  obtained  from  member  company  CAD/CAM 
systems  used  for  sheet  metal  die  design.  The  CDIM  SMI  Test 
Team  has  asked  the  PDES,  Inc.  tools  development  team  for 
support  of  the  following  additional  IGES  entities: 
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Primary  Importance:  142  - Curve  on  a Parametric  Surface 

144  - Trimmed  (Parametric)  Surface 

Secondary  Importance:  120  - Surface  of  Revolution 

140  - Offset  Surface 

• As  discussed  in  Section  4.3,  the  Parasolid  to  STEP  physical 
file  translator  developed  at  NIST  will  be  used  to  generate 
STEP  files  containing  B-Rep  data.  The  limitation  that  this 
translator  generates  STEP  physical  files  that  comply  with  the 
"Tokyo"  version  of  the  STEP  standard  may  reguire  that  software 
support  be  provided  by  PDES,  Inc.  to  update  this  file  to 
comply  with  the  latest  version  of  the  STEP  standard  (or  that 
specified  by  the  proposed  B-rep  Application  Protocol  AP204,  if 
appropriate ) . 

• Software  tool  reguirements  for  the  IDEFlX-based  portion  of 
CDIM  SMI  testing  have  not  been  further  defined  since  the 
initial  Tools  Reguirements  Document.  As  mentioned  in  Section 
4.4  of  this  document,  the  CDIM  Test  Team  still  plans  to 
perform  some  investigative  IDEF-based  testing  during  the  later 
stages  of  CDIM  SMI  testing.  For  this  reason,  software  tool 
support  from  PDES,  Inc.  may  be  reguested  to  support  this  work. 
The  CDIM  SMI  Test  Team  and  the  PDES,  Inc.  tools  team  would 
jointly  develop  more  detailed  reguirements  before  actual 
software  development  could  begin  on  these  IDEF-based  test 
tools . 

8 . 4 Training  Requirements 

CDIM  SMI  training  requirements  were  also  initially  addressed  in  the 
document , Tools  Requirements  for  CDIM  SMI  Development  and  Test 
[13].  This  document  basically  identified  that  training  was 
required  for  each  specific  software  tool  and  provided  the  expected 
method  of  training  (i.e.,  self-taught,  structured  class,  etc.)  and 
necessary  timeframe  when  known.  These  training  requirements  still 
apply  when  new  software  tools  are  introduced  into  the  program. 

Other  training  already  received  by  the  test  team  includes: 

• IDEF  and  Express  classes  provided  by  D.  Appleton  Company 
(DACOM) 

• Overview  education  in  the  application  areas  of  sheet  metal 
part  production  and  sheet  metal  die  design  (mainly  in  the  form 
of  reference  material  review,  production  facility  tours,  and 
application  expert  interviews) 

• Overview  training  in  the  Express-based  STEP  test  tools,  the 
UNIX  operating  system,  X-Windows  environment,  EMACS  text 
editor,  email.  Revision  Control  System  (RCS),  etc.  at  NIST 
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(Feb.  12-13)  and  SCRA  (Feb.  14) 

• CDIM  test  methodology  training  provided  by  Jack  Skeels 

• Conceptual  Modeling  training  provided  by  P.D.I.T.  for  both 
the  CDIM  SMI  Model  and  Test  Teams  (March  12-13) 

Additional  CDIM  SMI  Test  Team  training  requirements  include: 

• Structured  Query  Language  (SQL)  - A one-day  class  oriented 
toward  PDFS,  Inc.  activities  is  planned. 

• Training  in  CDIM  testing  methodology /philosophy  and  the  STEP 
testing  tools  will  be  ongoing  throughout  CDIM  SMI  testing. 
Most  of  this  training  is  very  informal  or  self-taught.  It  is 
expected  that  experienced  testers  from  past  PDES,  Inc.  CDIM 
projects  will  provide  guidance  and  informal  training  when 
necessary . 
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9.0  Test  Deliverables 


This  section  describes  the  planned  deliverables  from  the  CDIM  SMI 
Test  Team.  These  deliverables  include  the  CDIM  SMI  Test  Plan,  test 
procedure  documentation,  test  result  documentation,  test  issue 
reports,  a test  demonstration  capability,  and  the  CDIM  SMI  Testing 
Final  Report. 

9.1  CDIM  SMI  Test  Plan 

The  CDIM  SMI  Test  Plan  (i.e.,  this  document)  constitutes  the  first 
deliverable  of  the  CDIM  SMI  test  team.  This  document  outlines  the 
test  objectives,  constraints,  procedures,  activities,  resources, 
and  deliverables  of  the  test  team  throughout  CDIM  SMI  testing. 

9 . 2 Test  Procedure  Documentation 

Test  Procedure  Documentation  will  include  all  supporting 
information  that  defines  the  test  suites,  test  scenarios,  and  test 
cases  used  in  CDIM  SMI  testing.  Test  suite  documentation  will 
consist  of  a textual  overview  which  defines  the  company  perspective 
and  subject  area  domain.  Test  scenario  documentation  will 
primarily  consist  of  context  diagrams  with  company-specific 
definitions  for  the  activity  and  ICOMs.  Test  case  documentation 
may  consist  of  several  items,  including  portions  of  STEP  physical 
files,  design  drawings,  or  process  specifications,  and  will  include 
textual  information  to  describe  how  the  test  data  elements  are  used 
for  that  particular  test.  This  test  procedure  information  will  be 
documented  to  clearly  identify  the  CDIM  areas  that  are  tested. 
This  test  team  deliverable  is  primarily  support  documentation  to 
confirm  results  claimed  in  the  CDIM  SMI  Testing  Final  Report.  It 
is  important  to  note  that  raw  test  data,  such  as  design  drawings, 
process  specifications,  etc.,  are  not  distributed  past  individual 
testers  in  accordance  with  company  concerns  about  potentially 
sensitive  or  proprietary  data. 

9.3  Test  Result  Documentation 

Test  Result  Documentation  is  also  primarily  support  documentation 
to  confirm  results  claimed  in  the  CDIM  SMI  Testing  Final  Report. 
Test  results  and  other  test  information  will  be  recorded  by  each 
tester  while  performing  test  activities.  When  necessary,  test 
results  will  be  documented  on  CDIM  SMI  Test  Issue  Logs  (see 
Appendix  B) . Test  result  summary  information  will  be  provided  to 
the  CDIM  Model  Team  through  test  result  meetings  at  the  conclusion 
of  each  test/fix/test  iteration.  This  test  result  documentation  is 
recorded  to  identify  the  actual  extent  of  CDIM  testing  coverage. 
This  test  team  deliverable  is  primarily  a mechanism  for  sharing 
information  between  the  testing  and  modeling  teams  and  is  typically 
not  appropriate  for  extensive  outside  distribution.  The 
information  contained  in  this  documentation  is  also  summarized  in 
the  CDIM  SMI  Testing  Final  Report. 
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9.4  Test  Issue  Reports 

Test  Issue  Reports  are  a test  team  deliverable  that  are  provided 
primarily  to  the  CDIM  Model  Team.  These  issue  reports  will  be 
presented  to  the  model  team  in  a uniform  manner  through  the  CDIM 
SMI  Test  Issue  Log  (see  Appendix  B)  and  test  result  meetings  at  the 
conclusion  of  each  test/fix/test  iteration.  Further  information  on 
Test  Issue  Reports  is  provided  in  Section  10.0  of  this  document. 

9.5  Test  Demonstration  Capability 

As  identified  in  the  CDIM  SMI  Project  Plan  [16],  a final 
deliverable  from  the  testing  process  will  be  a test  demonstration 
capability.  This  demonstration  will  be  given  on  an  as-requested 
basis  and  will  consist  of  the  execution  of  several  meaningful  SQL 
queries  and  updates  performed  on  a database  populated  with  test 
data.  This  demonstration  is  intended  to  provide  Sheet  Metal 
Project  member  companies  with  tangible  results  from  the  CDIM  SMI 
effort. 

9.6  CDIM  SMI  Testing  Final  Report 

The  final  and  primary  test  team  deliverable  will  be  the  CDIM  SMI 
Testing  Final  Report.  This  report  will  include  the  following 
information: 

• An  overview  of  the  actual  test  activities,  procedures, 
facilities,  software  tools,  etc.  used  in  CDIM  SMI  testing. 

• A summary  of  the  test  results,  including  all  issues  (both 
resolved  and  outstanding) . 

• An  evaluation  of  the  actual  CDIM  testing  coverage,  including 
support  data  to  show  the  correlation  between  specific  test 
suites  and  the  CDIM  model . 

• An  overall  assessment  of  the  CDIM  model  in  terms  of  model 
thoroughness,  rigor,  completeness,  and  integrity. 

• An  overall  assessment  of  the  CDIM  testing  activity  in  terms 
of  the  test  objectives  stated  in  Section  3.2  of  this  test 
plan. 

• Documentation  on  any  "lessons  learned"  and  suggestions  for 
future  CDIM  testing  activities. 

• Final  recommendations  regarding  the  CDIM  data  model  and 
potential  Sheet  Metal  Project  follow-on  activities. 
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10.0  Issue  Procedures 


This  section  describes  the  procedures  for  reporting,  tracking,  and 
resolving  issues  discovered  during  CDIM  SMI  testing. 

10.1  Issue  Reporting 

Issues  that  are  generated  during  any  test/fix/test  iteration  will 
be  documented  using  the  CDIM  SMI  Test  Issue  Log  as  shown  in 
Appendix  B.  The  Test  Issue  Logs  will  be  a recording  and  reporting 
mechanism  for  the  test  team,  and  responsibility  for  maintenance  of 
these  logs  will  reside  with  the  test  team. 

Issues  can  be  generated  at  any  point  during  CDIM  SMI  testing  and 
can  be  written  against  any  aspect  of  the  testing  process,  including 
the  model  team  deliverables,  software  tools,  or  even  the  testing 
process  itself.  This  mechanism  allows  the  tester  to  document  any 
problem  encountered.  All  issues  will  first  be  addressed  at  the 
test  team  level  and  will  be  forwarded  to  the  appropriate  body  only 
after  team  approval.  Attempts  should  be  made  to  resolve  software 
or  testbed  problems  using  other  existing  mechanisms,  such  as  the 
NIST  PDES  Hotline,  if  appropriate,  before  generating  a Test  Issue 
Log. 

When  recording  an  issue,  all  available  supporting  data  should  be 
provided,  along  with  the  information  requested  in  the  Test  Issue 
Log.  This  information  may  include  items  such  as  the  testbed 
facility  or  software  tool  used,  the  activity  being  performed  when 
the  problem  was  encountered,  software  warning  or  error  messages,  or 
other  items  that  seem  relevant  to  the  issue.  All  test  issues 
against  the  CDIM  model  should  include  identifiers  to  the  specific 
test  suite,  test  scenario,  test  case,  and  appropriate  Activity/ICOM 
from  the  CDIM  SMI  Activity  Model. 

Test  issue  reports  can  also  optionally  include  test  team 
recommendations  for  resolving  the  issue.  Recommendations  will 
include,  where  appropriate,  an  explanation  and  sufficient  support 
data  to  justify  the  recommendation. 

10.2  Issue  Tracking 

Tracking  of  the  CDIM  SMI  Test  Issue  Logs  will  be  accomplished 
through  a Test  Team  Issue  Log  Custodian.  All  Test  Issue  Logs  will 
be  submitted  to  the  Issue  Log  Custodian  in  either  electronic 
(WordPerfect)  or  paper  form,  as  chosen  by  the  tester.  The  Issue 
Log  Custodian  will  maintain  an  Issue  Log  Notebook  which  includes 
paper  copies  of  all  test  issues. 

Test  issues  may  have  a status  of  either  OPEN,  CLOSED,  or  INACTIVE. 
A status  of  OPEN  means  that  the  issue  has  been  approved  as  an  issue 
by  the  test  team,  but  has  not  yet  been  resolved.  A status  of 
CLOSED  indicates  that  the  issue  has  been  resolved  to  the 
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satisfaction  of  the  test  team.  A status  of  INACTIVE  means  that  the 
issue  has  been  recognized,  but  a fix  is  currently  not  available  or 
will  not  be  addressed  by  current  work  items.  Test  issues  that  have 
been  reported  to  the  Issue  Log  Custodian,  but  not  approved  for 
release  by  the  test  team,  should  not  have  any  of  the  status  fields 
marked  on  the  Test  Issue  Log. 

In  addition  to  the  Issue  Log  Notebook,  the  Issue  Log  Custodian  will 
create  and  maintain  an  issue  log  listing  to  summarize  test  issue 
activity.  This  listing  will  be  a matrix  showing  the  creation  date, 
issue  number,  CDIM  version,  issue  status,  and  resolution  date.  The 
resolution  date  will  be  an  optional  field  used  only  for  those 
issues  that  have  been  closed. 

10.3  Issue  Resolution  Method 

Upon  validation  of  a Test  Issue  Log  by  the  test  team,  a copy  of  the 
issue  log  will  be  placed  in  the  Issue  Log  Notebook  and  the  issue 
log  listing  will  be  updated. 

At  this  point,  an  assessment  will  also  be  made  by  the  test  team 
whether  to  forward  the  issue  immediately  to  the  appropriate  body, 
or  to  wait  until  a significant  number  of  issues  have  been 
accumulated.  Although  the  answer  to  this  guestion  will  generally 
depend  on  the  severity  level  of  the  problem,  the  typical  approach 
for  handling  most  test  issues  will  be  to  deliver  all  new  issues 
together  at  specified  times  during  the  testing  process  (e.g., 
weekly,  on  selected  dates  established  with  the  model  team,  at  the 
conclusion  of  each  test/fix/test  iteration,  etc.). 

Resolved  issues  will  be  "closed"  through  either  a consensus 
agreement  by  the  test  team  and/or  by  verification  during  subseguent 
test/fix/test  iterations.  When  properly  resolved,  the  issue  status 
will  be  changed  to  CLOSED  and  the  issue  log  listing  will  be 
updated.  Both  resolved  and  outstanding  Test  Issue  Logs,  however, 
will  be  retained  for  later  inclusion  in  the  CDIM  SMI  Testing  Final 
Report . 
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Appendix  A:  Testing  Project  Schedule 


Note:  The  following  pages  contain  the  entire  CDIM  SMI  project 
schedule.  The  activities  on  this  schedule  which  pertain 
specifically  to  the  testing  effort  are  numbered  35  through  78. 
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6/5/91  7:26  am  SHEET  METAL  CDIM  SMI  Report  View:  Project  Tracking 
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6/5/91  7:26  am  SHEET  METAL  CDIM  SMI  Raport  View:  Project  Tracking 
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Jack.  A.  Skee^s 


Appendix  B:  CDIM  SMI  Test  Issue  Log 
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CDIM  SMI  TEST  ISSUE  LOG 


ISSUE  # CDIM  VERSION  DATE 

ORIGINATOR  NAME  

ACTIVITY  ICOM  

TEST  SUITE  TEST  SCENARIO  

TEST  CASE  TEST  RESULT  


ISSUE  DESCRIPTION: 


IMPACT:  CRITICAL  SEVERE  MINOR 

SUGGESTED  SOLUTION  (OPTIONAL): 


RESOLUTION  DATE:  RESOLVED  SY : 

RESOLUTION  DESCRIPTION: 


COMMENTS : 


ISSUE  STATUS:  OPEN  CLOSED  INACTIVE 
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Appendix  C:  Test  Suite  Example 
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The  following  diagram  is  an  IDEFIX  representation  of  the 
relationships  between  context  diagrams,  test  suites,  test 
scenarios,  test  data,  and  test  cases.  This  diagram  is  provided  for 
further  illustration  of  the  test  procedure  as  outlined  in  Section 
7.0  of  this  document.  Following  this  diagram  is  a sample  test 
suite  using  information  obtained  through  AE  workshops  held  with 
sheet  metal  die  design  experts  from  the  Boeing  company. 


scope 
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CDIM  SMI  - Boeing  Test  Suite  #1  (Example) 

Test  Suite  Overview 

This  test  suite  describes  the  Boeing  company  perspective  for 
testing  the  Manage  Die  activity  (A2.1)  from  the  CDIM  SMI 
Activity  Model.  See  attachments  for  the  CDIM  SMI  node  tree 
and  IDEFO  Activity  Model. 

The  following  context  diagrams  were  developed  during  AE 
workshops  with  Boeing  sheet  metal  experts  and  are  used  in  this 
test  suite: 

B2.1.1  - Commit  to  Schedule 
B2.1.2  - Conduct  Coordination  Meetings 
<list  all  context  diagrams> 

Test  Scenario  #1 


B2 . 1 . 1 - Commit  to  Schedule.  This  activity  consists  of  development 
of  a binding  contract  where  the  Tool  Design  group  agrees  to 
complete  a released  die  design  on  a given  schedule.  The  schedule 
is  determined  by  reviewing  preliminary  scheduling  information, 
manpower,  and  workload.  This  review  establishes  a schedule 
approval  and/or  modifies  a die  design  schedule. 


Manpower 

1 

j Workload 

1 I 

I I 

V V 


! Modified  Die 

Commit  to  Schedule  | — > Design  Schedule 

B2.1.1  } — > Schedule 

j Approval 


Die  Design  

Schedule 


ICOM  Definitions 


ICOM  - Die  Design  schedule 
DEFINITION 

Scheduling  information  for  a die  design  (GREEN  ROOM). 
COMPANY  SPECIFIC  DATA  SOURCES: 

Indentured  part  list  (IPL)  from  planning,  based  on  OLS 
data,  and  information  from  production  managers'  flow 
charts  (PMFC).  (IPL  and  PMFC  may  be  different,  depending 
if  tool  design  issues  are  present) . 
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ICOM  - Manpower 
DEFINITION 

Identifies  available  designers,  linked  to  workload. 
COMPANY  SPECIFIC  DATA  SOURCES: 

Designer  schedules. 

ICOM  - Workload 
DEFINITION 

Defines  die  design  committed  schedule  of  work,  linked  to 
manpower . 

COMPANY  SPECIFIC  DATA  SOURCES: 

Die  Design  work  schedule  information. 

ICOM  - Modified  die  design  schedule 
DEFINITION 

Modified  schedule  which  reflects  inability  to  meet  the 
production  part  schedule,  but  which  reflects  best  effort 
date,  based  on  current  work  load.  (GREEN  ROOM) 

COMPANY  SPECIFIC  DATA  SOURCES: 

Revised  production  manager  flow  chart  information,  or 
input  to  cause  the  revised  chart. 

ICOM  - Schedule  approval 
DEFINITION 

Approval  by  die  design  group,  which  reflects  acceptance 
of  planned  schedule.  (GREEN  ROOM) 

COMPANY  SPECIFIC  DATA  SOURCES: 

Signature  data  from  production  manager  flow  charts. 

Test  Case  #1 

This  test  case  addresses  the  Boeing  business  case  as  outlined 
below.  Specific  values  for  the  data  elements  are  taken  from  the 
Boeing  data  source  listed.  This  data  will  be  used  to  validate  the 
CDIM  data  model  using  the  testing  methodology  provided  in  Section 
4.0  of  the  CDIM  SMI  Test  Plan. 

ICOM  - Workload 

Die  Design  Work  Schedule 

line  1:  work  schedule  date  = 4/19/91 

line  5:  die  number  = ABC321 

line  12:  designer  name  = Larry  Jones 

line  13:  workload  % = 25 

line  20:  completion  date  = 6/19/91 

line  25:  % complete  = 0 

<insert  remaining  data  from  this  source> 

ICOM  - Manpower 

Designer  Schedules 

line  2:  designer  name  = Larry  Jones 
line  3:  department  = D/878 

line  4:  activity  name  = DESIGN  DIE  FACE  FOR  ABC321 
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line  5:  % time  = 25 

line  6:  start  date  = 5/1/91 

line  7:  end  date  = 6/19/91 

line  12:  activity  name  = CORRECT  PROBLEM  IN  DIE  DD4 
line  13:  % time  = 75 
line  14:  start  date  = 5/12/91 
line  15:  end  date  = ASAP  (estimated  5/15/91) 
<insert  remaining  data  from  this  source> 


ICOM  - Die  Design  Schedule 

Indentured  Part  List  (from  planning) 
line  14:  die  number  = ABC321 
line  16:  die  design  department  = D/878 
line  23:  reguired  completion  data  = 6/25/91 
line  26:  part  number  = AZ-76543-01 
line  28:  part  design  version  = D 
line  33:  part  designer  = John  Reynolds 

<insert  remaining  data  from  this  source> 


OLS  Data 
line 
line 
line 
line 
line 
line 
line 


3:  die  number  = ABC321 

4:  part  reference  number  = AZ-76543-01 

6:  last  modification  date  = 5/18/91 

9:  fiscal  year  = 1991 

14:  hours  logged  = 42 
18:  % complete  = 35 

19:  status  of  die  = working  prototype 
<insert  remaining  data  from  this  source> 


Production  Manager's  Flow  Charts 


line 
line 
line 
line 
line  8 
line  9 


flow  chart  ID  number  = XXY547 
date  of  flow  chart  = 5/22/91 
activity  name  = DESIGN  DIE  FACE  FOR  ABC321 
activity  start  date  = 5/01/91 
activity  end  date  = June  19,  1991 
person  responsible  = Larry  Jones 
<insert  remaining  data  from  this  source> 


ICOM  - Modified  Die  Design  Schedule 

Revised  Production  Manager's  Flow  Chart 


line 
line 
line 
line 
line 
line  8 
line  9 


flow  chart  ID  number  = XXY547 
name  of  production  manager  = Mary  R.  Smith 
date  of  flow  chart  = 5/25/91 
activity  name  = DESIGN  DIE  FACE  FOR  DIE  #5 
activity  start  date  = 6/01/91 
activity  end  date  = June  23,  1991 
person  responsible  = John  Jackson 
<insert  remaining  data  from  this  source> 


ICOM  - Schedule  Approval 

Production  Manager's  Flow  Chart  signature  data 
line  1:  flow  chart  ID  number  = XXY547 
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line  3:  name  of  production  manager  = Mary  R.  Smith 

line  4:  date  of  flow  chart  = 5/25/91 

line  33(a):  name(s)  of  approving  officer(s)  = 

Bill  Shaffer,  T.  Grume 

line  33(b):  approval  date(s)  = 5/28/91,  5/29/91 
<Insert  remaining  data  from  this  source> 


Test  Case  #1  Query  Strategy:  Write  gueries  based  on  the  structure 
of  the  CDIM  model;  then  check  if  they  match  the  context  boxes  and 
test  data. 

Test  Case  #2 

<Repeat  above  process  for  the  second  (and  all  subsequent)  test 
case(s)  in  Test  Scenario  #1,  if  applicable. > 


Test  Scenario  #2 

B2.1.2  - Conduct  Coordination  Meetings.  There  are  two  different 
types  of  coordination  meetings:  1)  Meetings  to  coordinate  all 

parties  concerned  with  producing  the  given  part/tool  and  initiating 
the  Tool  Design  Request.  This  meeting  is  schedule-oriented  (GREEN 
ROOM);  and  2)  Meetings  to  approve  the  die  design  concept.  A die 
design  schedule  review  is  performed  to  determine  the  die  production 
schedule  and  general  die  configuration.  Die  design  concept  ideas 
are  exchanged  to  determine  production  scheduling  indices.  This 
review  approves  or  modifies  the  die  design  schedule.  Once  this 
review  is  performed,  technical  meetings  are  held  to  determine 
concept  approval  or  identify  required  changes  to  either  the  die 
design  concept  or  the  part  design. 

<Insert  complete  B2.1.2  context  diagram  below. > 


V 


I 

V 


Conduct  Coordination  — > 

Meetings 

B2.1.2  — > 
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ICOM  Definitions 


<Insert  ICOM  names,  ICOM  definitions,  and  company  specific  data 
sources  for  this  context  diagram. > 


Test  Case  #1 

<Insert  test  case  description . > 

ICOM  - <Insert  ICOM  name> 

<Insert  name  of  data  source,  relevant  data  elements,  and 
data  values> 

<Repeat  above  process  for  all  ICOMs  in  this  context  diagram. > 
<Insert  Test  Case  Query  Strategy> 

Test  Case  #N 

<Repeat  above  process  for  all  test  cases  in  Test  Scenario  #2.> 
Test  Scenario  #N 

<Repeat  above  process  for  all  context  diagrams  in  the  test  suite. > 
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Appendix  D:  List  of  Acronyms 
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List  of  Acronyms 


AAM 

AE 

AP 

ARM 

B-Rep 

CAD 

CDIM 

DACOM 

EDS 

GEDM 

GPDM 

ICOM 

IDEFO 

IDEFIX 

IGES 

ISO 

ITOP 

NIST 

NPT 

OTON 

PC 

PDFS 

P.D. I .T. 

PDL 

PSCM 

QDES 

RCS 

SCRA 

SML 

SQL 

STEP 

STEPwf-SQL 


Application  Activity  Model 
Application  Expert 
Application  Protocol 
Application  Reference  Model 
Boundary  Representation 
Computer  Aided  Design 
Context  Driven  Integrated  Model 
D.  Appleton  Company 
Electronic  Data  Systems  Corp. 

Generic  Enterprise  Data  Model 
Generic  Product  Data  Model 
Input-Control -Output-Mechanism 

ICAM  Definition  Language  (for  activity  modeling) 
ICAM  Definition  Language  (for  data  modeling) 
Initial  Graphics  Exchange  Specification 
International  Organization  for  Standardization 
IGES  to  PDFS  (software  tool) 

National  Institute  of  Standards  and  Technology 
National  PDES  Testbed  (at  NIST) 

"Old"  to  "New"  (geometry  conversion  software  tool) 

Personal  Computer 

Product  Data  Exchange  Using  STEP 

Product  Data  Integration  Technologies,  Inc. 

PDES  Development  Lab  (at  SCRA) 

Product  Structure  Configuration  Management 

Quick  and  Dirty  Editor  for  STEP  (software  tool) 

Revision  Control  System 

South  Carolina  Research  Authority 

Structural  Modeling  Language 

Structured  Query  Language 

Standard  for  the  Exchange  of  Product  Model  Data 
STEP  Working  Form  to  SQL  (software  tool) 
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